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WHAT IS CLAIMED IS 



1 . A method for generating an enhanced compressed digital 
image, comprising the steps of: 

a) capturing a digital image; 

b) generating additional information relating to the importance of 
photographed subject and corresponding background regions of the digital image; 

c) compressing the digital image to form a compressed digital image; 

d) associating the additional information with the compressed digital 
image to generate the enhanced compressed digital image; and 

e) storing the enhanced compressed digital image in a data storage device. 

2. The method claimed in claim 1 , wherein the additional 
information generated in step (b) is further compressed to generate compressed 
additional information before the associating step. 

3. The method claimed in claim 2, wherein the step of further 
compressing the additional information employs a lossless compression technique. 

4. The method claimed in claim 1 wherein the digital image is one 
of a sequence of digital motion images. 

5. The method claimed in claim 1 wherein the step of 
compressing the digital image employs JPEG compression technique. 

6. The method claimed in claim 1 wherein the step of 
compressing the digital image employs JPEG2000 compression technique. 
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7. The method claimed in claim 1 wherein generating the 
additional information further comprises the steps of: 

b) collecting data from separate sensing elements, wherein all data is 
collected at the time of capturing the digital image. 

8. The method claimed in claim 1 wherein the additional 
information is a main subject belief map containing a continuum of belief values 
relating to the importance of the subject and background regions in the digital image. 

9. The method claimed in claim 8 wherein generating the main 
subject belief map comprises the steps of: 

al) extracting regions of homogeneous properties from the digital 

image; 

a2) extracting for each of the regions, at least one structural 
saliency feature and at least one semantic saliency feature; and 

a3) integrating the at least one structural saliency feature and the at 
least one semantic saliency feature using a probabilistic reasoning engine into an 
estimate of a belief that each region is the main subject. 

1 0. The method claimed in claim 9 wherein generating the main 
subject belief map further comprises the steps of: 

b) collecting data from separate sensing elements, wherein all data 
is collected at the time of capturing the digital image; and 

c) utilizing the collected data for modifying the estimate of the 
belief that each region is the main subject. 
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1 1 . The method claimed in claim 1 , further comprising the steps 

of: 

a) extracting the additional information from the enhanced 
compressed digital image; 

b) extracting the compressed digital image from the enhanced 

compressed digital image; 

c) decompressing the compressed digital image to form a 

decompressed digital image and; 

d) further compressing the decompressed digital image responsive 
to the additional information to generate a recompressed digital image not exceeding 
a target size. 

12. The method claimed in claim 1 1 where step d) further 

comprises the steps of: 

dl) performing a spatio-frequency transformation on the 
decompressed digital image to produce an array of transform coefficients; 

d2) deriving a distortion-weighting factor from the additional 
information for each transform coefficient; and 

d3) producing a recompressed digital image, not exceeding a target 
size, using an image compression system that is responsive to the distortion-weighting 
factors. 

1 3 . The method claimed in claim 7, further comprising the steps 

of: 

a) extracting the additional information from the enhanced 

compressed digital image; 

b) extracting the compressed digital image from the enhanced 
compressed digital image; 

c) decompressing the compressed digital image to form a 
decompressed digital image; 
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d) generating a main subject belief map from the extracted 
additional information and the decompressed digital image; and 

e) further compressing the decompressed digital image responsive 
to the main subject belief map to generate a recompressed digital image not exceeding 
a target size. 

14. The method claimed in claim 13 where step e) further 

comprises the steps of: 

e 1 ) performing a spatio-frequency transformation on the 
decompressed digital image to produce an array of transform coefficients; 

e2) deriving a distortion-weighting factor from the belief map for 
each transform coefficient; and 

e3) producing a recompressed digital image, not exceeding a target 
size, using an image compression system that is responsive to the distortion-weighting 
factors. 

15. A method for recompressing a decompressed digital image 
using a main subject belief map to generate a recompressed digital image, comprising 
the steps of: 

a) performing wavelet decomposition on the decompressed digital 
image to produce an array of wavelet coefficients that can be used to reconstruct the 
decompressed digital image by summing corresponding synthesis basis functions 
weighted by the wavelet coefficients; 

b) deriving a distortion-weighting factor from the belief map for 
each transform coefficient; and 

c) producing a recompressed digital image not exceeding a target 
size from the wavelet coefficients using the distortion-weighting factors to minimize 
an error function. 
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16. The method claimed in claim 15 where step b) further 
comprises the steps of: 

bl) for a wavelet coefficient, identifying a region of support in the 
belief map corresponding to that wavelet coefficient; and 

b2) calculating a distortion-weighting factor for the wavelet 
coefficient using belief values in the identified region of support. 

17. The method claimed in claim 15, wherein the recompressed 
digital image is JPEG2000 compliant 

18. The method claimed in claim 1 5, wherein the error function is 
a mean square error function weighted by the distortion-weighting factors. 

19. The method claimed in claim 16, wherein the step b2) of 
calculating a distortion-weighting factor for the wavelet coefficient is performed by 
averaging the belief values in the region of support and calculating the distortion- 
weighting factor as a function of the average belief value. 

20. The method claimed in claim 4, further comprising the steps 

of: 

a) extracting the additional information from each enhanced 
compressed digital image in the sequence; 

b) extracting the compressed digital image from each enhanced 
compressed digital image in the sequence; 

c) decompressing each compressed digital image to form a 
sequence of decompressed digital images; and 

d) compressing each decompressed digital image from the 
sequence, responsive to the additional information, to generate a sequence of 
recompressed digital images not exceeding a target size. 
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21 . he method claimed in claim 4 wherein generating the additional 
information for each digital image in the sequence further comprises the steps of: 

b) collecting data from separate sensing elements, wherein all data 
is collected at the time of capturing the digital image. 

22. The method claimed in claim 21 , further comprising the steps 

of: 

a) extracting the additional information from each enhanced 
compressed digital image in the sequence; 

b) extracting the compressed digital image from each enhanced 
compressed digital image in the sequence; 

c) decompressing each compressed digital image to form a 
sequence of decompressed digital images; 

d) generating a main subject belief map for each digital image 
from each decompressed digital image and the corresponding extracted additional 
information; 

e) recompressing each decompressed digital image, responsive to 
the main subject belief map, to generate a sequence of recompressed digital images 
not exceeding a target size. 

23. The method claimed in claim 22, wherein the step e) of 
recompressing each decompressed digital image in the sequence further comprises the 
steps of: 

a) dividing the decompressed digital image into a plurality of 

regions; and 

b) determining a quantization parameter for each of the regions as 
a function of the main subject belief values for that region. 

24. The method claimed in claim 23, wherein the regions are 
macroblocks and the sequence of recompressed digital images is MPEG compliant. 
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25. The method claimed in claim 23, wherein the regions are 
macroblocks and the sequence of recompressed digital images is H.263 compliant. 

26. The method claimed in claim 23, wherein step b) of 
determining the quantization parameter is performed by averaging the main subject 
belief values in the region and calculating the quantization parameter as a function of 
the average belief value. 

27. The method claimed in claim 1 wherein the step of generating 
additional information includes the steps of: 

a) calculating a representation of a relative strength of high- 
frequency components in regions of the image; and 

b) compressing and encoding the representation of the relative 
strength of high-frequency components for inclusion with the compressed digital 
image. 

28. The method claimed in claim 1 wherein the step of generating 
the additional information comprises the steps of: 

a) tracking eye movement, during capture, using eye tracking 
sensors in a camera viewfinder; 

b) generating gaze-tracking data from the eye tracking sensors; 

and 

c) encoding the gaze-tracking data. 

29. The method claimed in claim 1 wherein the step of generating 
the additional information comprises the steps of: 

a) capturing a depth map of objects in each frame using a depth 

sensor; 

b) generating depth data from the depth sensor; and 
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c) encoding the depth data. 



30. A system for generating an enhanced compressed digital image, 

comprising: 

a) means for compressing a digital image to form a compressed 

digital image; 

b) means for generating additional information that relates to a 
photographed subject's importance with regard to the captured digital image, and 
corresponding background regions of the captured digital image; 

c) means for weighing the additional information relative to the 
photographed subject's importance with regard to the captured digital image such that 
weighted additional information is produced; and 

d) means for associating the weighted additional information with 
the compressed digital image to produce the enhanced compressed digital image. 

31. A system for transcoding an enhanced compressed digital 
image, comprising: 

a) means for extracting additional information from the enhanced 

compressed digital image; 

b) means for extracting a compressed digital image from the 
enhanced compressed digital image; 

c) means for decompressing the compressed digital image to form a decompressed 
digital image and; 

d) means for further compressing the decompressed digital image 
responsive to the additional information to generate a recompressed digital image not 
exceeding a bit stream target size. 



